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ABSTRACT 

In  adapting  C2  to  the  21st  century  we  plan  to  conduct  a  controlled  Human- In-  The-  Loop 
(HITL)  experiment  with  new  Network  Centric  Warfare  (NCW)  technology  which  will  be 
introduced  to  sixteen  experienced  warfighters  in  the  form  of  a  collaborative  User 
Defined  Operational  Picture  (UDOP)  with  Blue  Force  Readiness  and  intelligence 
database  access  enabled  by  an  IP  wide  area  network  as  a  possible  improvement  over 
their  use  of  current  baseline  technology  in  the  form  of  the  GCCS  with  Common 
Operational  Picture  (COP)  capability.  We  examine  here  the  general  methodology  of 
using  controlled  HITL  experiments  employing  combat  scenarios  as  an  Operational 
Assessment  for  testing  and  evolving  more  effective  C2  technology  for  the  warfighter. 
(See  the  Award-winning  TTCP  GUIDEx,  2006) 

Our  general  theoretical  hypothesis  views  the  warfighting  team  as  a  Complex  Adaptive 
System  (CAS),  and  asserts  that  a  team’s  use  of  shared  informational  schema  contributes 
to  their  shared  mental  models  resulting  in  increased  combat  effectiveness  in  the 
battlespace.(See  Gell-Mann,  1994,1997)  In  particular,  we  hypothesize  that  the  results  of 
this  experiment  will  show  significant  improvements  on  the  NCW  performance  metrics  of 
Situational  Awareness,  Shared  Situational  Awareness  and  bottom-line  Combat 
Effectiveness  due  to  use  of  the  new  NECC(Net-Enabled  Command  Capability) 
technology  employed  in  the  experiment  trials.  The  especially  important  role  of  enhanced 
operational  replanning  quality,  and  speed,  enabled  by  the  new  technology,  will  be 
carefully  examined  here,  since  recent  experimentation  results  suggest  them  as  NCW 
metrics  that  warrant  more  scrutiny  by  the  research  community.  (See  Hiniker  &  Entin, 
2006).  Thus  we  expect  that  collaboration  and  synchronized  replanning  will  play 
important  roles  impacting  combat  effectiveness  in  this  C2  experiment. 
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INTRODUCTION 


Net  Centric  Warfare  (NCW)  has  been  defined  as  an  information  superiority-enabled 
concept  of  operations  that  generates  increased  combat  power  by  networking  sensors, 
decision  makers,  and  shooters  to  achieve  shared  awareness,  increased  speed  of  command, 
higher  tempo  of  operations,  greater  lethality,  increased  survivability,  and  degree  of  self  - 
synchronization.  (Alberts,  Garstka,  and  Stein  (2000))  Situational  Awareness  (SA),  as 
well  as  its  sharing  by  linked  warfighters  (SSA),  is  thus  deemed  to  be  a  major  causative 
factor  in  increasing  combat  power.  (Hiniker  &  Entin,  1990,  1992:  Perry,  et  al,  2004; 
Hiniker,  2005;  Hiniker  &  Entin,  2006)  Increased  Speed  of  Command,  and  the  associated 
increased  speed  and  quality  of  planning,  have  recently  received  some  empirical  support 
as  NCW  contributors  to  combat  effectiveness.  (Hiniker  &  Entin,  2006)  Besides  the 
higher  connectivity  created  through  the  construction  of  broader  band  networks,  the  major 
information  technologies  that  are  indispensable  for  enabling  NCW  for  a  warfighting  team 
are  a  Common  Operational  Picture  (COP)  coupled  with  a  shared  whiteboard  for 
collaboration  over  the  map  of  the  battlespace.  DISA’s  most  advanced  versions  of  these 
technologies  are  the  User  Defined  Operational  Picture  (UDOP),  as  instantiated  by  NECC 
C2  Common  Services,  combined  with  NCES  e-collab  whiteboard  with  shared  map 
planning  and  voice  functionality.  In  addition,  warfighter  access  to  remote  data  base 
capabilities  such  as  a  Blue  Force  Readiness  database  and  an  improved  Red  Force  Tracker 
database  should  contribute  directly  to  the  speed  and  quality  of  replanning,  and  hence  also 
contribute  to  increased  NCW  combat  effectiveness. 

Does  greater  Speed  of  Command  via  improved  speed  and  quality  of  replanning  utilizing 
remote  data  bases  by  a  distributed  warfighting  team  enabled  by  a  network  in  fact  cause 
improved  combat  effectiveness?  What  are  some  of  the  causal  mechanisms  involved? 
The  purpose  of  the  experiment  described  here  is  to  demonstrate  and  to  analyze  the 
differences  in  operational  effectiveness  between  current  warfare  practices  and  NCW 
practices  using  the  combined  technologies  of  the  UDOP  with  collaboration  technology, 
and  the  improved  remote  intelligence  and  blue  force  readiness  databases  while  capturing 
quantitative  measures  of  NCW  parameters  under  controlled  conditions.  Here 


collaboration  technology  is  instantiated  through  the  NCES  e-collab  with  shared  map 
planning  capability  and  audio. 

In  addition,  this  experiment  will  serve  as  an  early  Operational  Assessment  in  the  Test 
and  Evaluation  of  a  combination  of  Capability  Modules  (CM)  as  contributors  to  the 
effectiveness  of  the  Adaptive  Planning  Capability  Definition  Package  (CDP)  which  is 
expected  to  increase  significantly  the  measures  of  the  Key  Performance  Parameters  of  the 
Combat  Loss  Exchange  Ratio  and  the  Speed  and  Quality  of  Replanning  and  Shared 
Situational  Awareness  in  the  combat  mission  threads  played  out  by  the  participating 
warfighting  teams  here  when  compared  to  their  performance  with  the  current  baseline 
technology.  (See  DoD/DAU,  2003;  TTCP  GUIDEx,  2006)  Here  the  subset  of  CMs 
utilized  in  the  experiment  will  be  drawn  from  the  mature  members  of  the  following 
collection:  UDOP/iCOP  (User  Defined  Operational  Picture);  TMS/CWS  (Red  Force 
Tracker);  Blue  Ground  Force  Tracker;  Weather;  and  SORTS  (Blue  Force  Readiness).  All 
of  these  CMs  are  considered  to  be  shared  Schema  that  may  contribute  to  more  complete 
and  more  accurate  representations  inside  the  warfighting  team  of  the  relevant  aspects  of 
the  external  combat  environment. 

Such  early  Operational  Assessments  of  prototypes  are  important  to  the  efficient 
evolutionary  development  of  C2  technology.  In  the  development  of  C2  technology  it  is 
useful  to  benchmark  progress  through  the  use  of  standard  measures  of  performance  and 
effectiveness.  Thus  use  of  the  standard  performance  parameter  of  Situational  Awareness 
was  made  in  a  1990  experiment  (Hiniker  &  Entin,  1990)  and  again  in  2006  in  an 
experiment  with  a  similar  scenario  and  set-up.  The  comparison  of  the  results  of  these  two 
experiments  demonstrated  noteworthy  evolution  in  C2  capability  for  the  warfighter  over 
the  intervening  decade  and  a  half  (Hiniker  &  Entin,  2006).  The  baseline  condition  in  the 
first  experiment  consisted  in  local  tactical  pictures  located  at  the  two  ship  captain  posts 
and  a  big  picture  Gulf  view  located  at  the  remote  team  leader’s  command  post.  This 
experiment  showed  significantly  higher  Situational  Awareness  by  the  warfighters  in  the 
COP  prototype  treatment  condition.  The  later  experiment  used  the  COP  condition  as 
baseline  and  found  comparable  and  significant  improvements  in  Situational  Awareness  in 
the  post-replanning  phases  of  the  experiment  when  warfighters  employed  the  new 
collaborative  UDOP  technology.  Similar  significant  improvements  were  found  for 


bottom-line  Combat  Effectiveness  when  comparing  experiment  results  over  the  decade. 
(Hiniker  &  Entin,  1992)  Such  comparisons  in  the  evolution  of  technological  progress  in 
C2  would  be  more  difficult  to  draw  without  the  use  of  such  standard  measurements  of 
operational  performance  parameters.  (See  Measurement  Appendix) 

In  the  current  experiment,  we  examine  the  impact  of  these  surviving  collaborative  UDOP 
technologies  coupled  with  new  network  enabled  distributed  intelligence  and  readiness 
databases  while  focusing  on  the  measurement  of  their  contribution  to  effective  adaptive 
planning  by  the  warfighting  teams.  In  this  paper,  we  will  first  present  our  experimental 
approach  to  evaluating  the  new  C2  technology.  Next  we  describe  the  specific  warfighting 
scenarios  employed  and  the  NCW  metrics  to  be  taken  during  them.  Then  we  present  a 
broad  Complex  Adaptive  Systems  (CAS)  view  of  NCW,  including  the  role  of  schema, 
and  derive  our  experimental  hypotheses.  Finally,  we  describe  the  analysis  methods  to  be 
used  for  testing  the  hypotheses  with  the  metric  data  obtained  during  the  experiment  trials 
and  the  kinds  of  conclusions  one  may  draw  from  the  findings. 

APPROACH 

The  NECC  Delta  Experiment  introduced  above  is  another  "true  experiment”  with 
controls  examining  the  effectiveness  of  some  new  C2  technology,  drawn  from  NECC, 
utilized  by  a  distributed  command  team  of  joint  warfighters  collaborating  and  replanning 
over  an  IP  Network  with  access  to  remote  operational  planning  data  bases  while 
engaged  in  simulated  combat  scenarios  compared  to  the  effectiveness  of  warfighter 
performance  employing  current  baseline  condition  technology.  In  the  approach  adopted 
here,  variably  equipped  warfighting  teams  are  experimentally  created  in  a  controlled 
Human-In-The-Loop  (HITL)  experiment  utilizing  the  JTLS  wargame  simulator,  and 
their  performances  are  systematically  related  to  combat  outcome.  (See  TTCP  GUIDEx, 
2006)  In  the  experiment  each  of  four  warfighting  teams,  each  composed  of  an  Air 


Force  officer,  a  pair  of  Naval  Tactical  Action  Officers  (TAO)  and  an  ashore  higher 
level  operational  planner  at  CJTF  with  the  authority  to  change  Rules  of  Engagement 
(ROE)  and  provide  additional  blue  forces,  play  out  four  battles  composed  of  modified 
versions  of  a  Persian  Gulf  air/sea  counter-terrorist  combat  scenario,  termed  Operation 
Storm  Petrel  II,  crossed  with  two  versions  of  information  technology,  the 
collaborative  UDOP  with  associated  distributed  intelligence  databases  and  the 
baseline  GCCS  COP/CHAT  ensemble.  The  baseline  technology  is  that  currently  in  use 
by  most  of  our  forces  for  scenarios  similar  to  ours. 

The  mission  of  the  blue  teams,  including  the  two  ship  captains  protecting  two  major  and 
six  minor  oil  platforms  off  Basrah  in  the  Persian  Gulf,  is  to  identify  and  to  prosecute 
advancing  terrorist  fast  attack  craft  and  pirated  aircraft.  In  each  scenario  run,  one  of  the 
Naval  officers  plays  the  role  of  captain/TAO  of  the  guided  missile  destroyer  USS 
Winston  S.  Churchill  with  all  its  resources  (e.g.  guns,  missiles,  helicopter),  the  other 
Naval  officer  plays  the  role  of  captain/TAO  of  the  guided  missile  destroyer  USS  Mason 
with  all  its  resources  and  the  Air  Force  officer  plays  the  role  of  Air  Operations 
Coordinator  (AOC)  -  controlling  all  fixed  wing  blue  aircraft  in  the  simulation.  A  joint 
staff  officer  plays  the  role  of  the  ashore  higher  level  operational  planner  at  CJTF  who 
provides  changes  to  the  readily  available  additional  blue  forces,  assistance  with  changes 
to  the  team  plan,  and  timely  alerting  and  delivery  of  relevant  intelligence  and  changes  to 
the  ROEs  over  the  net.  Opposing  these  blue  forces  are  a  dozen  terrorist  fast  attack  craft, 
Boghammers,  and  two  pirated  Cessnas  or  two  stolen  MIG  29s  under  red  control  as  played 
by  a  JTLS  simulator  operator.  There  are  also  two  other  blue  ships,  USS  Arctic  and  USS 
Ardent,  and  dozens  of  neutral  ships  and  commercial  aircraft  in  the  area.  Each  of  the  four 
slightly  modified  scenarios  is  divided  into  three  time  phases:  TP1  consisting  of  Stage 
Setting  and  initial  combat  operations;  TP2  consisting  of  Replanning  triggered  by  a 
surprising  new  terrorist  assault  against  oil  platforms  or  US  Naval  ships,  involving  red  fast 
attack  craft  or  pirated  aircraft,  or  an  abrupt  change  in  the  ROEs  from  higher  blue 
authority;  and  TP3  consisting  of  the  End  Game  of  the  combat  operation.  The  basic 


scenario  is  analogous  to  the  Basrah  terrorist  incident  of  Spring  2004  as  well  as  to 
Operation  Praying  Mantis  of  1986,  and  bears  some  similarities  to  the  March  2007  Iranian 
capture  of  15  British  sailors  off  Basrah.  (See  Fig.  1  below) 


Figure  1.  UDOP  Screenshot  of 
^^Operation  Storm  Petrel  Scenario 


NCW  Metrics.  For  all  four  newly  created  four-man  teams  and  for  each  time  phase  of  all 
four  trials,  Situational  Awareness  (SA)  is  defined  as  the  proportion  of  the  crisis  relevant, 
or  mission  critical,  set  of  warfighting  platforms  in  the  battlespace,  red,  blue  or  neutral, 
correctly  identified  as  important  by  the  commander.  (Hiniker  &  Entin,  1990;  Hiniker, 
2002;  Perry  et.al,  2004;  Hiniker,  2005;  Hiniker  &  Entin,  2006))  During  the  simulated 
combat  operation,  using  the  JTLS  wargame  simulator,  the  commander’s  realization  of  the 
situation,  his  Situational  Awareness,  is  obtained  by  his  drawing  on  a  map  the  platforms 
he  deems  important  at  that  time,  i.e.  by  his  personal  Cognitive  Operational  Graphic 
(COG).  The  commander’s  proportion  correct  is  then  obtained  by  comparison  with  the 
platforms  on  the  simulator’s  Ground  Truth  map  at  the  same  time.  Greater  overlap 
between  the  commander’s  COG  and  Ground  Truth  is  indicative  of  greater  Situational 
Awareness  by  the  commander  at  the  time.  Greater  overlap  between  the  COGs  of  the  team 
of  three  commanders  is  indicative  of  greater  Shared  Situational  Awareness  (SSA). 


(Hiniker,  2002;  Perry  et  al,  2004;  Hiniker  &  Entin,  2006)  Thus  these  SA  metrics  take 
account  of  the  fit  for  each  mission  relevant  weapons  platform  between  its  psychological 
world  cognition,  its  information  world  record  and  its  physical  world  ground  truth 
existence.  Such  COG  measures,  together  with  a  set  of  additional  measures  including 
current  Plan  quality  and  teamwork  are  obtained  by  trained  observers  for  all  commanders 
at  the  end  of  each  of  the  three  phases  of  each  of  the  four  combat  scenarios. 

Planning  quality  (P),  itself,  is  measured  by  summing,  and  then  averaging  for  the  team, 
the  seven-point  Likert  scale  observer  evaluations  of  five  items  comprising  the  quality  of 
performance  of  the  stages  of  the  OODA  Loop  planning  cycle:  observation,  orientation, 
decisionmaking,  execution,  and  overall  planning  and  plan  execution  performance. 
(Hiniker  &  Entin,  2006) 

Speed  of  Command  (td)  is  measured  by  summing  the  team’s  time  to  size  up  the  situation 
plus  time  to  replan  plus  time  to  act  plus  time  to  complete  the  decision  cycle  with  battle 
damage  assessment  and  begin  to  review  the  new  situation.  (See  Measurement  Appendix) 
Each  replanning  cycle  begins  with  a  surprising  new,  unanticipated  move  by  red  or  with  a 
higher  level  ordered  change  in  the  blue  ROEs.  Thus  Speed  of  Replanning,  tr ,  is  the  time 
taken  within  the  new  decision  cycle  between  the  team’s  newly  sizing  up  the  changed 
situation  and  the  team’s  beginning  the  new  course  of  action. 

Finally,  the  combat  Effectiveness  (E)  of  the  warfighting  team  for  each  time  phase 
derives  from  the  JTLS  wargame  simulator  tally  of  the  loss  exchange  ratio  of  warfighting 
platforms  for  the  time  phase,  red  losses/  red  plus  blue  plus  neutral  losses.  (Hiniker,  1991; 
Hiniker  and  Entin,  1992;  Hiniker  &  Entin,  2006) 

During  the  experiment  all  teams  operate  as  “edge  organizations”  in  that  command  is 
relatively  decentralized,  team  member  interactions  are  relatively  unconstrained,  and 
information  is  broadly  distributed.  What  differentiates  the  teams  is  the  technology  they 
use.  Four,  four-man  joint  warfighting  teams  will  prosecute  the  Operation  Storm  Petrel  II 
scenario  during  16  counter-balanced  trials  conducted  in  summer  2007  as  a  Limited 
Objective  Experiment  (LOE)  at  the  JFCOM  Joint  Systems  Integration  Command 


laboratory  under  two  different  technology  treatment  conditions: 


•  In  the  C2  Baseline  treatment  condition,  all  four  military  players  share  the  same 
GCCS  COP  view  of  the  Gulf  and  communicate  via  CHAT,  using  the  current 
intelligence  product  obtained  under  current  time  lines.  This  is  the  technology  suite 
with  which  most  of  our  players  have  operational  experience. 

•  In  the  new  NCW  treatment  condition,  the  airman  receives  track  and  intelligence 
data  injects,  and  C2  common  services  tailored  to  air  Community  of  Practice,  the  two 
sea  captains  receive  track  and  intelligence  data  injects  and  C2  common  services 
tailored  to  maritime  Community  of  Practice,  and  the  higher  level  planner  at  CJTF 
receives  all  of  these  data  plus  access  to  the  Blue  Force  Readiness  data  base  and  all 
four  warfighters  jointly  collaborate  in  combat  using  a  common  NCES  e-collab 
whiteboard  with  drawing  functionality  and  with  VOIP  and  joined  Operational 
Context  with  stated  Commander’s  Intent  forming  a  common  Community  of  Action. 
Here  all  communications  for  the  distributed  team,  including  Operation  Storm  Petrel 
web  portal  access,  information  searches,  and  simulator  and  operational  data  base 
updates,  are  conducted  over  an  IP  broadband  wide  area  network. 

A  CAS  VIEW  OF  NCW  AND  EXPERIMENT  HYPOTHESES 

It  is  useful  to  conceive  of  the  warfighting  team,  either  distributed  or  local,  as  a  Complex 
Adaptive  System  (CAS).  The  collaborative  UDOP  with  associated  distributed 
intelligence  databases  provide  shared  schema  for  the  warfighting  team,  i.e.  shared 
representations  inside  the  warfighting  team  of  the  relevant  external  environment. 
(Hiniker,  2002)  These  shared  representations  or  “schema”  provide  the  CAS  with 
descriptions,  predictions,  and  prescriptions  for  effective  interactions  in  the  environment. 
(Gell-Mann,  1994;  Gell-Mann,  1997)  The  UDOP  represents  the  current  situation;  and  the 
e-collab  whiteboard  permits  shared  graphic  representation  of  future  planned  situations  in 
the  battlespace.  The  intelligence  databases  provide  up-to-date  information  on  the 
location  of  red  warfighting  platforms,  i.e.  red  combat  elements;  and  the  readiness 
database  provides  information  on  the  current  readiness  status  of  blue  warfighting 
platforms,  both  local  and  global.  Such  informational  schema,  representing  the  relevant 


aspects  of  the  situation  and  what  to  do  about  it,  form  the  major  portion  of  the  relevant 
message  traffic  passed  around  the  communications  system,  and  taken  together  these 
messages  constitute  replicas  of  the  state  of  the  command  decision  process,  itself.  (See 
Girard,  1990)  For  the  most  part,  in  this  “information  world”,  observations  and 
assessments  come  in  and  go  up;  plans  and  directives  come  down  and  go  out.  (See 
Alberts,  Garstka  &  Stein,  2000  for  a  useful,  recent  explication  of  the  distinctions  between 
the  informational,  cognitive  and  physical  domains  in  C2  research) 

The  shared  schema  when  internalized  by  human  warfighters  constitute  shared  mental 
models  (Rouse  and  Morris,  1986),  and  should  enable  the  warfighting  team,  conceived  of 
as  a  unified  CAS,  to  complete  the  group  OODA  (Observe-Orient-Decide-Act)  Loop 
process  more  rapidly  and  effectively  leading  to  greater  combat  effectiveness.  UDOP 
schema  should  mainly  aid  SA  of  the  battlespace;  e-collab  shared  whiteboard  schema 
should  mainly  aid  Planning  activities  in  the  battlespace:  and  distributed  intelligence  and 
readiness  databases  contribute  to  both  sets  of  processes.  Internalized  schema  have 
proved  useful  to  effective  human  action.  Klein,  for  example,  has  demonstrated  that 
schematic  representations  of  prototypical  situations  are  often  directly  associated  with 
scripts  that  produce  single  step  retrieval  of  actions  from  human  memory,  thus  prompting 
rapid  “recognition-primed  decisionmaking.”  (See  Klein  &  Salas,  2001). 

A  mental  model,  or  internalized  schema,  is  a  symbolic  representation  which  may  take  one 
of  two  basic  forms,  linguistic  representations  and  discourse  models  (See  Chomsky, 
1962).  Discourse  models  make  explicit  the  structure  not  of  sentences  but  of  situations  as 
we  perceive  or  imagine  them  (Johnson-Laird  1983,  p.  419).  Indeed  pure  linguistic 
representations  do  not  say  anything  about  how  words  relate  to  the  world;  whereas  a 
discourse  mental  model,  or  “picture”  model  in  the  conception  of  Wittgenstein  (1922), 
represents  the  reference  of  discourse,  i.e.  the  situation  that  the  discourse  describes.  The 
COP,  for  example,  provides  such  a  “picture”  model  that  should  be  an  aid  to  a  team’s 
sharing  accurate  discourse  models  of  the  battlespace  (Hiniker,  1998).  Whereas  there  may 
be  many  different  discourse  models  available  in  a  warfighting  team,  for  a  given 
individual  at  a  given  point  in  time  his  “Situation  Awareness”  reflects  the  current  state  of 
his  mental  model  of  the  situation  (Endsley  2000,  p.  12).  Endsley,  working  primarily  with 
fighter  pilots  and  air  traffic  controllers,  has  demonstrated  that  better  “situation 


awareness”  is  “probabilistically  linked”  to  better  performance.  (See  also  Hiniker  &  Entin, 
1990)  In  short,  mental  models  of  the  situation  and  goals,  e.g.  “Commander’s  Intent”  or 
team  mission,  help  select  what  information  from  the  welter  in  the  environment  is  attended 
to  and  thereby  help  increase  the  likelihood  of  better  decisions  and  better  performance  of 
action. 

Since  the  informational  schema  are  shared  as  mental  models  by  human  warfighters,  their 
effectiveness  is,  of  course,  subject  to  the  human  cognitive  constraints  of  bounded 
rationality,  analogous  to  the  effects  of  channel  capacity  constraints  on  the  speed  of 
information  transmission  over  a  network:  Performance  impairing  information  overload 
can  and  does  occur  at  both  the  cognitive  and  the  informational  levels  of  a  socio-technical 
system.  (Levis  et  al,  1987;  Hiniker,  2002)  Finally,  at  the  “ground  truth”  level  of  the 
physical  world,  as  well,  human  actors,  sensors,  weapons  platforms,  communications 
networks,  and  associated  software  and  data  bases  can  and  do  become  impaired  in  the 
course  of  warfare.  (See  Fig.  2  below) 


Figure  2.  Three  Viewpoints  on  Elements  in  the 

Battlespace 


INFORMATIONAL 


Shared  Situational  Awareness 
of  ship  in  Gulf 


COGNITIVE 


COP  Schema  record 
of  ship  in  Gulf 


PHYSICAL 

Ground  Truth  presence 
of  ship  in  Gulf 


Thus  the  interactions  of  the  schema-sharing  CAS  with  its  environment  entail  both  simple 
linear  and  complex  non-linear  relationships.  Human  information  overload  is  an  instance 
of  a  non-linear  relationship  in  that  a  small  positive  change  in  informational  workload  near 
the  cognitive  crash  threshold  results  in  a  very  large  degradation  in  performance. 
Furthermore,  new  incoming  information  to  a  human  actor  is  not  necessarily  automatically 
believed  and  internalized  as  part  of  the  mental  model;  if  the  new  information  is  dissonant 
with  current  belief  it  may  be  dismissed,  denigrated  or  otherwise  modified.  (See 
Festinger,  1957)  Finally,  most  network  interactions  involving  humans  are  not  simple 
random  network  interactions;  rather  they  usually  involve  “small  world”  nets  including 
shortcuts  or  “scale  free”  nets  including  hubs  and  may,  under  certain  conditions,  exhibit 
non-linear  “percolation  effects”  (Moffat  and  Atkinson,  2005)  Here  the  focus  will  be 
upon  the  existence,  rather  than  the  form,  of  causal  relationships  between  a  warfighting 
team’s  use  of  shared  informational  schema,  contributing  to  their  shared  mental  models, 
and  the  consequent  effectiveness  of  their  operations  in  the  battle  space.  (See  Pearl,  2001) 
Several  causal  hypotheses  regarding  expected  empirical  relationships  are  proposed 
below. 

Hypothesis  1.  By  facilitating  the  development  of  more  accurate  and  more  complete 
shared  mental  models,  use  of  the  collaborative  UDOP  with  associated  distributed  data 
bases  by  a  warfighting  team  causes  significant  improvement  in  their  Situational 
Awareness  (SA).  (See  Hiniker  &  Entin,  1990).  This  effect  should  be  amplified  in 
scenarios  in  which  the  assessment  is  highly  uncertain,  i.e.  situations  in  which  there  are 
many  ambiguous  fast  moving  tracks  of  potential  mission  relevance.  The  tailored  expert 
views  afforded  by  UDOP  coupled  with  the  broader  channel  for  team  communication 
provided  by  the  e-collab  shared  whiteboard  coupled  with  audio  should  help  mitigate  the 
information  overload  when  compared  with  use  of  the  baseline  COP/CHAT  technology. 


Access  to  the  distributed  intelligence  and  readiness  databases  should  contribute  to  the 
accuracy  of  their  Situational  Awareness. 


Hypothesis  2.  By  facilitating  information  sharing  and  group  consensus  on  the  important 
and  relevant  weapons  platforms  in  the  situation,  use  of  the  collaborative  UDOP  with 
associated  distributed  databases  by  a  warfighting  team  causes  significant  improvement  in 
Shared  Situation  Awareness  (SSA)  across  the  team.. 

Hypothesis  3.  By  facilitating  information  sharing  and  group  consensus,  use  of  the 
collaborative  UDOP  with  associated  distributed  databases  by  a  warfighting  team 
increases  the  quality  or  desirability  of  their  developed  Plan  (P). 

Hypothesis  4.  This,  in  turn,  increases  the  synchronicity  of  the  warfighting  team’s  action 
(A),  leading  to  greater  Combat  Effectiveness  (E). 

Hypothesis  5.  Use  of  the  collaborative  UDOP  with  associated  distributed  databases 
should  increase  the  speed  with  which  the  warfighting  team  typically  completes  the 
OODA  Loop  (t<j),  including  the  speed  of  replanning,  tr ,  also  leading  to  greater  Combat 
Effectiveness  (E).  (See  Appendix  for  measurement  definitions  of  terms) 

RESULTS 

The  analysis  of  the  results  of  hypothesis  testing  in  this  experiment  will  utilize  the 
techniques  of  Analysis  of  Variance  (ANOVA),  following  a  counter-balanced,  within 
subjects  design,  to  determine  whether  or  not  use  of  the  collaborative  UDOP  with 
associated  distributed  databases  technology  enabled  the  warfighting  teams  to  perform 
significantly  more  effectively  on  the  NCW  performance  metrics  examined  when 
compared  to  their  use  of  the  baseline  technologies.  Use  of  these  quantitative  performance 
metrics  also  permits  one  to  estimate  the  values  of  the  NCW  parameters  exhibited  by  the 
warfighting  teams,  under  each  of  the  treatment  conditions.  In  addition  to  these  objective 
performance  measures,  subjective  evaluations  by  the  participants  of  the  two  sets  of 


technologies  will  also  be  solicited  and  analyzed.  We  intend  to  use  a  modified  version  of 
the  standard  JDCAT  measuring  instrument  for  the  purposes  of  gathering  such  warfighter 
opinion 


CONCLUSIONS 

In  this  controlled  HITL  experiment  we  plan  to  introduce  new  NCW  technology  to 
sixteen  experienced  warfighters  in  the  form  of  a  collaborative  User  Defined  Operational 
Picture  with  associated  distributed  intelligence  databases  accessible  over  a  wide  area 
network  to  enable  a  possible  improvement  over  their  performance  with  current  baseline 
technology.  The  findings  from  this  experiment  will  have  implications  in  three  major 
areas:  the  possibility  of  testing  causal  hypotheses  on  the  effectiveness  of  new  information 
technology  for  the  warfighter  in  a  controlled  laboratory  environment;  the  related 
possibility  of  developing  a  program  systematically  to  evolve  more  effective  information 
technology  that  accumulates  greater  capability  over  time;  and  the  possibility  of 
developing  more  general  theory  on  the  determinants  of  effective  behavior  of  warfighting 
teams  viewed  as  Complex  Adaptive  Systems,  involving  men  and  machines. 

First  of  all,  we  have  demonstrated  with  this  HITL  Lab  experiment  and  several  earlier 
experiments  ( Hiniker  &  Entin,  1990,  1992  &  2006)  that  one  can  test  causal  hypotheses 
on  the  impact  of  new  information  technology  for  creating  significantly  more  effective 
combat  decisionmaking  by  warfighters.  (TTCP,  2006) 

It  follows  from  the  above  that  such  HITL  experimentation  can  be  useful  in  the 
evolutionary  development  of  more  effective  information  technology  for  the  warfighter 
We  can,  and  have,  built  upon  and  compared  our  evolving  new  technology  with  a  rising 
baseline  of  improved  technology  that  leads  to  demonstrably  improved  operational 
performance  over  the  decades.  In  particular,  we  have  developed  C2  technology  which 
significantly  improved  the  warfighters’  Situational  Awareness  and  technology  which 


significantly  improved  the  warfighters’  measured  combat  Effectiveness.  To  increase  the 
efficiency  of  developing  such  technology,  one  can  usefully  employ  these  methods  of 
controlled  experimentation  with  prototypes  as  early  Operational  Assessments  for  Test 
and  Evaluation  of  emerging  C2  technologies.  (DoD/DAU,  2003)  The  results  of  our 
last  warfighting  experiment  are  encouraging  in  this  regard.  They  showed  significant 
improvements  in  Situational  Awareness,  Shared  Situational  Awareness  and  bottom-line 
combat  Effectiveness  across  several  phases  of  the  scenario  played  out  by  the  teams  due 
to  use  of  the  new  collaborative  UDOP.  (Hiniker  &  Entin,  2006).  The  superiority  of  the 
collaborative  UDOP  technology  over  the  current  baseline  technology  on  these  NCW 
metrics  was  especially  pronounced  in  the  post-replanning  phases  of  the  scenario. 
Furthermore,  the  quality  of  Planning,  itself,  when  warfighters  were  using  the 
collaborative  UDOP,  was  consistently  and  significantly  superior  to  that  using  the 
baseline  technology  for  all  phases  of  all  scenarios.  The  current  planned  experiment, 
involving  network  access  by  the  warfighters  to  additional  distributed  planning  databases, 
should  build  upon  this  demonstrated  advantage  provided  by  the  collaborative  UDOP 
technology  for  improved  combat  planning  by  a  distributed  warfighting  team. 

Finally,  Complex  Adaptive  Systems  theory,  explicitly  involving  informational  schema, 
offers  an  illuminating  approach  to  these  information  technology  development  issues. 
Better  schema,  i.e.  internal  models  that  describe,  predict  and  prescribe  actions  in  the 
battlespace,  when  shared  and  internalized  by  members  of  a  warfighting  team,  should 
make  for  greater  combat  effectiveness.  It  seems  self-evident  that  a  more  complete  and 
accurate  picture  of  the  battlespace  will  better  describe  it  and  thereby  enable  more 
effective  combat  action  by  a  warfighting  team  that  shares  the  picture.  Indeed, 


experimental  evidence  shows  that  COP,  and  now  the  advanced  UDOP,  yield  improved 

Situational  Awareness  and  improved  combat  Effectiveness.  What  is  not  so  obvious  is 

that  the  type  of  shared  schema,  whether  a  linguistic  model  or  a  discourse  (picture)model, 
should  make  an  important  difference  to  the  team’s  combat  Effectiveness.  The  results  we 

have  obtained  so  far  strongly  suggest  the  superiority  of  a  shared  discourse  model  of  the 

battlespace,  e.g.  COP  or  UDOP,  over  a  shared  linguistic  model,  e.g.  information  relayed 

via  CHAT,  especially  when  it  comes  to  quality  operational  Planning  by  the  warfighting 

team.  We  await  more  experimental  evidence  for  more  definitive  answers  to  such 

questions. 


APPENDIX 

Measurement  Definitions  for  Collaborative  UDOP  Replanning  Delta  LOE 


Situational  Awareness  (SA)  =  Proportion  of  mission  critical  set  of  warfighting 
platforms  in  the  battlespace  correctly  identified  by  a  warfighter  (Ground  Truth  cf. 
Cognitive  Operational  Graphic  (  COG)  @  ti) 

Shared  Situational  Awareness  (SSA)  =  Proportion  of  overlap  between  pairs  of 
COGs  for  complete  warfighting  team. 

-  Plan  Quality  (P)  =  Accuracy  of  knowledge  of  scheduled  sequence  of  blue  moves. 
(See  text  on  Likert  scale  planning  process  evaluation  scoring  method) 

Speed  of  Command  (td  =  tc  +  tr  +ta  +  tb  ),  where  total  speed  of  command  is  the  sum 
of  time  to  size  up  situation  +  time  to  plan  +  time  to  act  +  time  to  complete  decision 
cycle  with  battle  damage  assessment 

Combat  Effectiveness  (E)  =  Loss/Exchange  Ratio=  red  platform  losses  /  (red  +  blue 
+  neutral  losses) 


Subjective  Opinion  of  Operational  Value  of  Technology  =  Participants’  scoring  of 
value  of  the  technology  on  seven  point  Likert  scale. 
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Delta  Experiment  Purpose 


•  To  discover  and  to  demonstrate  the  differences(  A)  in  operational 
effectiveness,  on  NCW  metrics,  between  current  warfare  practices 
using  present  Command  and  Control  technologies,  and  new 
Network  Enabled  Command  Capability  (NECC)  practices  using  the 
combined  DISA  capabilities  of  the  User  Defined  Operational 
Picture  (UDOP)  with  associated  net  enabled  remote  intelligence 
data  bases  and  blue  force  readiness  data  bases  coupled  with 
collaboration  technologies  instantiated  by  Net  Centric  Enterprise 
Services  (NCES)  white  board  with  shared  map  planning  and  voice 
functionalities. 


Scenario  Overview 


•  Persian  Gulf  setting  where:  Operation  Storm  Petrel  II  involves 


-  Two  Blue  Ships  (DDGs),  JFACC  air  &  ashore  CJTF  planner 

-  w /  changing  ROEs  protecting  several  oil  platforms  under 
attack  by: 

•  Twelve  Red  fast  attack  crafts,  Zhuks  and  Boghammers,  w / 
surprising  moves 

-  Analogous  to  the  Basrah  terrorist  incident  of  Spring 
2004 

•  Pirated  Aircraft 


DIS£  ^C2  Baseline  vs.  NCW  Technology 


•  In  C2  baseline  condition,  all  four  military  players  share  a  COP 
view  of  the  Gulf  and  communicate  via  internet  relay  chat. 
Intelligence  products  are  obtained  under  current  time  lines 
(e.g.  via  hard-copy  message,  l&W  briefings). 

•  In  NCW  condition,  JFACC/AOC  subscribe  to  air  track  and  intel 
OpContext  for  air  Community  of  Practice,  and  two  Navy 
destroyers  subscribe  to  maritime  track  and  intel  OpContext 
for  maritime  Community  of  Practice,  the  ashore  CJTF 
operational  planner  subscribes  to  all  these  and  blue  force 
readiness  data  and  all  jointly  collaborate  over  wide  area 
network  using  common  shared  white  board  with  stated 
Commander’s  Intent  forming  a  common  Community  of  Action. 
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UDOP  Screen  Shot  of  Operation 
Storm  Petrel  Scenario 
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v  A  CAS  View  of  Warfighting 


•  A  warfighting  team  can  be  usefully  considered  to  be  a  CAS 

•  Complex  Adaptive  Systems  (CAS)  possess  schema 

•  A  shared  schema  is  a  shared  representation  inside  the  warfighting 
team  of  the  relevant  external  environment 

•  Warfighters  use  of  shared  schema  improves  their  shared  mental 
models 

•  The  state  of  a  warfighter’s  mental  model  of  the  situation  at  any 
particular  time  is  indicated  by  his  Situational  Awareness 

•  More  accurate  and  more  complete  shared  mental  models  cause 
more  effective  combat  action  by  a  warfighting  team 


Hypotheses 


Warfighter  use  of  collaborative  UDOP  with 
associated  intel  and  blue  force  readiness 

schema  causes: 

(HI)  increased  Situational  Awareness(SA); 
(H2)  increased  Shared  SA(SSA); 

(H3)  increased  Planning  Quality(PQ); 

(H4)  increased  Combat  Effectiveness(CE); 

(H5)increased  Speed  of  Replanning^,.). 


□ISA 


T  &  E  Outputs 


•  Early  Operational  Assessment:  Uses  HITL  simulation  to 
evaluate  operational  effectiveness  and  potential  risks  of 
prototypes. 

*  Evolutionary  Development  driven  by  Engineered  Mission 
Threads: 
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Adaptive  A  Loss  Exchange  Ratio  e.g.  CMX 

Planning  (MOE);  +CMy 

A  Speed  of  Replanning  +CM7. 
(MOP)  * 

A  Shared  Situational  Awareness 

*  See  Measurement  Definitions  in  Appendix 


H3:  Collaborative  UDOPw /  dB  Causes 
Increased  Planning  Quality — Example  Data 
- Presentation 


ary  of  Expected  Significant  Findings 


•  Warfighters’  Hot  Wash  Inputs 

•  Warfighting  Teams’  Measured 
Performance  in  Simulated  Combat 
Trials 

•  Conclusions  on  Effectiveness  of  New 
Tech  Compared  to  Baseline  Tech. 


Backups 


MSA 


Network  Centric  Warfare 


•  A  promise  of  Network  Centricity: 
Unprecedented  operational  tempo  and 
situational  awareness  through  networked 
connectivity 

-  Ability  to  support  collaborative  environments 


DIS/Uas  Views  of  the  Battlespace 


Three  Viewpoints  on  Elements  in  the  Battlespace 


COGNITIVE 

Shared  Situational  Awareness 
of  ship  in  Gulf 


INFORMATIONAL 

COP  Schema  record 
of  ship  in  Gulf 


PHYSICAL 

Ground  Truth  presence 
of  ship  in  Gulf 


Dependent  Measures 
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JTLS  Screenshot  of  Storm  Petrel  Scenario 
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Measurement  Definitions  for  Operational  Assessment  of 
aborative  UDOP  w/lntel  and  Blue  Force  Readiness  Databases 


Confidence  Intervals  for  ASA  (Ax)  from  exp,  for  t-distribution 
Ax  -  ta  (s/Vn)  <  [i  <  Ax  +  ta  (s/Vn),  where  s  =V  (EXj  /n-1 ) 

Confidence  intervals  for  ASA  (Ax)  from  exp,  for  F-distribution 
(X.1-X.2)  -  VFasw  V(2(k-1)/n)  <  H  <  (x x  2)  +  a/Fusw  V(2(k-1)/n), 
where  sw  =  V(wss/k(n-1))  and  wss  =  within  groups  sum  of  squares 

Situational  Awareness  (SA)  =  Proportion  of  mission  critical  set  of  warfighting 
platforms  correctly  identified  by  a  warfighter  (Ground  Truth  cf.  COG  @  tj) 

Shared  Situational  Awareness  =  Proportion  of  overlap  between  pairs  of 
COGs  for  complete  warfighting  team. 

Speed  of  Command  (td  =  tc  +  tr  +ta  +  tb  ),  where  total  speed  of  command  is  the 
sum  of  time  to  size  up  situation  +  time  to  plan  +  time  to  act  +  time  to 
complete  decision  cycle  with  battle  damage  assessment 

Combat  Effectiveness  =  Loss/Exchange  Ratio=  red  platform  losses  /  (red  + 
blue  +  neutral  losses) 


HI:  Collaborative  UDOPw/dB  Causes  Increased  SA 
--Example  Data  Presentation 
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